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ABSTRACT 



This document highlights the problems with current federal 
education program data collection activities and lists several factors that 
maxe mu/emcut tovrard a possible solution, then discusses the vision ice me 
Integrated Performance and Benchmarking System (IPBS) , a vision of an 
Internet -based system for harvesting information from states about federal 
program activities. In discussing the IPBS vision, the document addresses two 
questions: How would students' privacy rights and data confidentiality be 
ensured? and How could the IPBS benefit the U.S. Department of Education (ED) 
and its customers? The next section focuses on the ED-sponsored two-state 
(Nebraska and Oregon) test of concept to explore the feasibility of the 
vision of the IPBS. Information is provided in terms of: goals; how the test 
of concept coordinated with the National Center for Education Statistics; 
program policy questions addressed; types of data collected; the big picture 
of how data got to the IPBS; how data was actually collected; reports that 
the test includes; and where to view the results 

(https://secure.air.org/ipbs). A final section outlines the main objectives 
in six phases through September 2004. (AEF) 



Reproductions supplied by EDRS are the best that can be made 
from the original document. 
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Current U.S. Department of Education (ED) program data collection activities 
labor under several problems that affect the quality and usefulness of program 
data. Most notably, the data obtained through the system often do not provide 
necessary policy information. In this regard, 

□ Many policy questions concerning program outcomes cannot be 
answered by existing program data collections. 

□ By the time the data are prepared and disseminated, they are no 
longer current and up-to-date. Consequently they are not useful 
to policy makers and education officials. 

□ A large number of data collections impose substantial burden on 
state and local education. 

D The disconnection between ED policies and data requests creates 
unnecessary duplication of requested data elements. 



Several factors now make movement toward a solution possible: 

□ Congress expects more timely and higher-quality performance 
data about programs for policy decisions and Government and 
Performance and Results Act (GPRA) reports. 

□ Many states are engaged in substantial efforts to shift to 
electronic data systems. By revising its data systems, ED can help 
insure that its systems are compatible with systems the states are 
developing to provide high-quality, comparable, and timely data. 

□ ED needs to move toward implementing the Government 
Paperwork Elimination Act, which requires that all record- 
keeping should, insofar as possible, be in electronic form by 2003. 

□ Many elementary and secondary education programs are in the 
process of reauthorization and will therefore need to revise their 
data collection activities. 



EXISTING DATA 
COLLECTIONS CANNOT 
ANSWER MANY POLICY 
QUESTIONS 
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The Integrated Performance and Benchmarking System (IPBS) is a vision of an 
Internet-based system for harvesting information from states about federal 
program activities at the school and district level. The system would allow 
users to link federal program participation and outcome information to 
characteristics of recipient states, districts, and schools. The IPBS could help to 
meet policymakers’ need for timely, outcome-based information while 
streamlining, modernizing, and reducing the reporting burden on states of federal 
information requests. 

The IPBS relies on a new approach to federal-state exchange of information 
about federal program outcomes. States will no longer send data to the federal 
government. Rather, states will collect and store the data in their own 
warehouses in such a way that the federal government can harvest them. States 
will monitor and ensure the quality of district- and school-collected data. 

Specifically, the IPBS is 



AN INTERNET-BASED 
SYSTEM FOR HARVESTING 
INFORMATION ABOUT 
FEDERAL PROGRAM 
ACTIVITIES 



D A shared set of core data and performance indicators. ED and 
states will work together to identify key policy questions about 
program outcomes and characteristics of program recipients. 

□ An electronic data harvesting system designed to minimize the 
burden on states. The IPBS will harvest data from state 
administrative records and be designed to be compatible with 
modern state data systems. 

□ A program database resource shared by ED and states. Reports 
could include a national education report card displaying progress 
on key ED Strategic Plan performance objectives, and annual 
performance reports. 



How Would Students' Privacy Rights 
and Data Confidentiality Be Ensured? 

The IPBS would be designed with care to ensure the complete privacy of 
individuals and confidentiality of sensitive data in the following ways: 

D No information about individual students, teachers, or parents 
would be collected, maintained or otherwise used in the system. 

D No data would be included in the IPBS system until they are 

reviewed and approved for inclusion by the state education agency. 

D State data warehouses would retain electronic confidentiality and 
privacy firewalls around data not designated for inclusion in IPBS. 
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BENEFITS FOR CONGRESS 



0 Timely, high-quality program performance data for policy 
decisions and GPRA reports. 

□ Reduction in data collection burden on states. 

□ Timely, accurate, state, and school district profile reports. 

□ Information on the relationship between program outcomes and 
federal spending. 



BENEFITS FOR DEPARTMENT OF EDUCATION POLICYMAKERS 

□ Timely data about federal program results for ED’s Strategic Plan 
and Annual Report. 

□ School- and district-level data elements allow for flexible analysis. 



BENEFITS FOR DEPARTMENT OF EDUCATION PROGRAMS 

□ More timely and higher-quality data. 

D Up-to-date information about program recipients and their 
characteristics. 

□ Ability to run custom program research reports on or find 
program recipients with particular characteristics. 



TIMELY, HIGH-QUALITY 
PROGRAM PERFORMANCE 
DATA 



BENEFITS FOR STATES 

□ Information on the relationship between program outcomes and 
federal spending. 

□ Reduced burden as states will report IPBS data once for use by 
multiple ED programs. 

□ Partner with ED on program data collection. 
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ED sponsored a two-state test of concept to explore the feasibility of the vision 
of the IPBS. This two-state test of concept was undertaken in partnership with 
the Council of Chief State School Officers (CCSSO) and the states of Nebraska 
and Oregon. American Institutes for Research, with KPMG, conducted the 
two-state test of concept. 

The objectives of the two-state test of concept were: 



To assess the feasibility of the IPBS on a small scale by harvesting 
current-year program performance data from two state data 
warehouses via the internet and using it to create a Web site and 
database. 

To assess the types of technical assistance states may require to 
modify their data warehouses and data collection methods to be 
able to participate in data harvesting. 

To use the experiences gained from the two-state trial to better 
estimate the required costs, time, and challenges involved in 
developing a full-scale IPBS. 

-SS feasibility, we started with two states with substantially different 
data systems. The test of concept version of the IPBS includes data on all 
districts and schools in Nebraska, with the exception of achievement data, which 
are available only for Title I schools. The IPBS test of concept also includes data 
from the 15 districts and 325 schools that are participating in Oregon’s Data 
Base Initiative. 

We focused on a few data elements from several elementary and secondary 
federal education programs, in particular: 



A PARTNERSHIP AMONG 
ED, CCSSO, AND THE 
STATES 



To asst 



□ Title I 

D Eisenhower Professional 
Development Program 

□ Goals 2000 

□ Safe and Drug-Free Schools 

□ Impact Aid 



□ IDEA Part B 

D Technology Literacy 
Challenge Fund 

□ McKinney Flomeless Assistance 
D Title VI 
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The test of concept version of the IPBS provides useful information on 
programs that 

□ Are administered by K-12 education agencies; and 
0 Serve many districts and/or schools in each state. 

The test of concept version of the IPBS can be useful to 

0 Provide information on school and district background 
characteristics; 

0 Track changes in gross quantitative outcomes; and 

D Provide program targeting, participation, and expenditure 
information. . 

The test of concept version does not work as well for 

D Programs with only a few recipients; 

0 Programs targeted to individual students; or 
0 Programs that go to non-school entities. 

Further development of the IPBS might address these limitations. 



tVJILESTOWES 

Pre-October 1999 
October 18, 1999 
December 7, 1999 



February 3, 2000 



Preliminary meetings 

Start of two-state test of concept 

Demonstration of preliminary 
Web site at meeting of all partners in 
Nebraska 

Official completion of two-state test 
of concept 







THE IPBS TEST OF 
CONCEPT BENEFITED 
FROM NCES' 
EXPERTISE 



The IPBS test of concept has benefited from NCES’ experience and coordinated 
its efforts with NCES by 

0 Learning from the National Forum as a model for how IPBS 
could work with states about federal program data. 

0 Using CCD identifiers and importing some CCD data elements. 
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0 Using NCES data definitions from the Student Data 
Handbook and Staff Data Handbook . 

0 Adapting NCES 5 On-Line Data Analysis system for custom 
report generation. 



The two-state test of concept version of the IPBS permits examining four 
main types of questions about federal programs. 

STUDENT OUTCOMES 

0 How do outcome trends in schools participating in various 
federal programs compare with trends in similar schools not 
participating in these programs? 

0 How have student outcomes, as measured by statewide 
assessments, changed over time in high-poverty schools? In 
high-minority schools? In schools with high proportions of 
LEP students? In urban schools? 



0 How well are federal programs targeted to high-poverty 
schools? For example, what percent of the highest-poverty 
schools (schools with more than 75% of students eligible for 
free or reduced lunch) are receiving Title I funds? What 
percent are participating in schoolwide programs? 

EDUCATIONAL QUALITY 

0 What is the trend in the ratio of computers to students? How 
does the trend differ for students in high-poverty and other 
schools? 

0 What is the trend in percent of teachers with less than a 
master s degree? How do these trends differ for big city and 
other schools? 



0 What percent of district expenditures is federal, state, and 
local? How do these percents differ by district poverty? 
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The IPBS test of concept harvested the following types of data FROM states 
ABOUT districts and schools. This list is NOT exhaustive, but illustrative. 

No individual student data were collected, maintained, or otherwise used. 

IDENTIFIERS 

Name and location (district name, state name, CCD#) 



Federal program participation 

Amount of federal funds received, by program 

STUDENT CHARACTERISTICS 
Enrollment 
Enrollment by grade 

Number of students, by race/ethnicity by grade 
Number of students, by poverty (free, reduced-price lunch) 
Number of students with disabilities 
Number of LEP students 

STUDENT PERFORMANCE 

Percent of students at each achievement level on 
statewide and district assessments 

Graduation rates 

Dropouts 

TEACHERS 

Number of teachers, by education 





The intent of the IPBS is not to replace or otherwise directly affect state data 
warehouses. States, districts, and schools will maintain whatever current data 
systems they have. Therefore, the two-state test of concept needed to provide a 
secure way for data from each state to be translated into a common format and 
transmitted to the IPBS database. The keys in the diagram below represent 
IPBS security links along the path through which information flowed. Once 
data were housed in the IPBS database, they became available for use by states 
as well as by federal program managers. 
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Three steps characterize how data went from state records to a common format 
usable by state and federal data users: 

D Data harvesting; 

□ Loading into the IPBS; and 

D Reporting. 

States posted a file containing pointers to data within their data systems or 
actual data to a location of their choosing on their Web site. The IPBS “data 
harvester” polled the designated Web sites for this file at predetermined times. 
When the IPBS found an updated file, it harvested it via an encrypted internet 
protocol. Once received by the IPBS, automatic data validation procedures 
checked to ensure that the data were within specified parameters. Any problems 
were noted and an error message returned to the state via email. States then 
corrected and reposted for harvesting. This process continued until all data met 
the IPBS reporting parameters. At this point, states were given a final opportunity 
to approve the data as harvested. States retained this crucial responsibility of 
validating the data the IPBS harvested. State validated data were then loaded 
into the IPBS database. 

Once in the IPBS database, the data are available to state and federal data 
users for a variety of purposes, most notably: 

D GPRA reporting; 

□ Program support and improvement; and 

□ Online data analysis. 



States 
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State Data Users 



ABILITY TO RUN CUSTOM 
REPORTS TO ANSWER 
HIGHLY SPECIFIC 
RESEARCH QUESTIONS 



The IPBS two-state test of concept included a number of predefined reports 
as well as a custom report generator, including: 

□ Annual State by State Performance on ED Strategic Plan 
Objectives. This report provides a quick summary of how well 
schools within each state and across both states are meeting 
selected Strategic Plan objectives and indicators. 

□ Student Achievement on State Assessments, Selected 
Comparisons and High Poverty Schools. While no measure of 
student achievement that is comparable across states has yet 
been implemented fully, these reports display student 
achievement on state assessments. 

□ Program Participation. These reports allow for the comparison 
among recipients of federal education aid. One focuses on the 
characteristics of districts and the other on the characteristics of 
schools receiving program funds, including several measures of 
student outcomes, instructional quality, and educational 
context. 

□ Program Funding. This report for selected elementary and 
secondary programs details the amount of federal funding per 
state. 

□ Proportion of Education Revenues Provided by Various 
Sources. This report illustrates the relative share of funding 
sources (federal, state, local) of education across states. 

□ Custom Reports Using On-Line Data Analysis. The IPBS test 
of concept Web site allows users to query the IPBS database to 
generate custom reports to answer highly specific research 
questions. This reporting mechanism is only available in a 
limited fashion for the IPBS test of concept. Future increments 
of the IPBS will include increasingly robust custom reporting 
features. 
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Participants 
What's in the 



iPBS 
What's New 
Browser Issues 



Welcome lolhc Integrated Performance Benchmarking 
System (IPBS) Project home page. 

The IPBS isn test of concept erf a vision of an 
Internet-baMd Kyittem for horvestlngioformstioa 
from Mot (K about federal program activities at the 
school and district level. The system would allow users to 
link federal program participation and outcome 
in form at ion to char ncteristicsof recipient states, 
districts, and schools. 



If you are new to this site, you may gptothc About the Protect page togti a brief introduction to the project. If you 
have a quest ion, goto the Contacts nape for project team email addresses and phone numbers. Thanks for visiting, 
and feel free to give us feedback! 



Look for the an ic 
Rentier 



a tdcmifyingdocumcntson this site that use the PDF format. They require: Adobe Acrobat 



For best results, you will want to view this site usingthe latest versionsof Internet Explorer or Netscape. To 
download the latest versionsof their browsers, use the icons to gptothc desired site: mm 



If you have quest ions or comments about the IPBS Website, please scnde-innil to webmaster. 
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PHASE 0: 



EXPAND PARTNERSHIP 
BETWEEN ED AND THE 
STATES 



Develop IPBS Concepts — Explore Feasibility With 

Two-State Troal, April 1 999-Febrmary 2000 

D Completed. 

PHASE II: 

Review and Revise Two-State Trial — Possibly Expand to 

Eights Sttaties, SVIarch-December 2000 

D Convene focus groups to assess feasibility of moving toward a real 
IPBS. 

D If the test of concept is deemed a success and interest continues, 
convene a Steering Committee of stakeholders. 

D Secure funding for future activities. 

D Expand partnership to additional interested states. 

D Thoroughly review and revise content and procedures used in test 
of concept and revise as appropriate for wider implementation. 

D Plan indicators and data elements to be included in the system. 

D Begin work on hard-to-measure variables, including student 
achievement, technology, and teacher quality. 



PHASE III: 

Revise amid Expamid to 25 States, 
Jamiiuiary-Decemniber 2001 



PHASE IV: 

Revise amid Expamid to ADD States amid Tenrltoiiies, 
Jammairy-Decemrabeir 2002 



PHASE V: 

Solidify Systemra lira All States amid Prepare for Traraslttlora fromm 
Old Systemra to IMew IPBS, 

Jamwiary-Decemraber 2003 



PHASE VI: 

FojiDDy Fuimicttlomilmig IPBS, 
Jamiiuiary-Septomniber 2004 
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